<!DOCTYPE html>
<html>

<head>

<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title>RedisCluster搭建Redis集群</title>


<style type="text/css">
body {
  font-family: Helvetica, arial, sans-serif;
  font-size: 14px;
  line-height: 1.6;
  padding-top: 10px;
  padding-bottom: 10px;
  background-color: white;
  padding: 30px; }

body > *:first-child {
  margin-top: 0 !important; }
body > *:last-child {
  margin-bottom: 0 !important; }

a {
  color: #4183C4; }
a.absent {
  color: #cc0000; }
a.anchor {
  display: block;
  padding-left: 30px;
  margin-left: -30px;
  cursor: pointer;
  position: absolute;
  top: 0;
  left: 0;
  bottom: 0; }

h1, h2, h3, h4, h5, h6 {
  margin: 20px 0 10px;
  padding: 0;
  font-weight: bold;
  -webkit-font-smoothing: antialiased;
  cursor: text;
  position: relative; }

h1:hover a.anchor, h2:hover a.anchor, h3:hover a.anchor, h4:hover a.anchor, h5:hover a.anchor, h6:hover a.anchor {
  background: url() no-repeat 10px center;
  text-decoration: none; }

h1 tt, h1 code {
  font-size: inherit; }

h2 tt, h2 code {
  font-size: inherit; }

h3 tt, h3 code {
  font-size: inherit; }

h4 tt, h4 code {
  font-size: inherit; }

h5 tt, h5 code {
  font-size: inherit; }

h6 tt, h6 code {
  font-size: inherit; }

h1 {
  font-size: 28px;
  color: black; }

h2 {
  font-size: 24px;
  border-bottom: 1px solid #cccccc;
  color: black; }

h3 {
  font-size: 18px; }

h4 {
  font-size: 16px; }

h5 {
  font-size: 14px; }

h6 {
  color: #777777;
  font-size: 14px; }

p, blockquote, ul, ol, dl, li, table, pre {
  margin: 15px 0; }

hr {
  background: transparent url() repeat-x 0 0;
  border: 0 none;
  color: #cccccc;
  height: 4px;
  padding: 0;
}

body > h2:first-child {
  margin-top: 0;
  padding-top: 0; }
body > h1:first-child {
  margin-top: 0;
  padding-top: 0; }
  body > h1:first-child + h2 {
    margin-top: 0;
    padding-top: 0; }
body > h3:first-child, body > h4:first-child, body > h5:first-child, body > h6:first-child {
  margin-top: 0;
  padding-top: 0; }

a:first-child h1, a:first-child h2, a:first-child h3, a:first-child h4, a:first-child h5, a:first-child h6 {
  margin-top: 0;
  padding-top: 0; }

h1 p, h2 p, h3 p, h4 p, h5 p, h6 p {
  margin-top: 0; }

li p.first {
  display: inline-block; }
li {
  margin: 0; }
ul, ol {
  padding-left: 30px; }

ul :first-child, ol :first-child {
  margin-top: 0; }

dl {
  padding: 0; }
  dl dt {
    font-size: 14px;
    font-weight: bold;
    font-style: italic;
    padding: 0;
    margin: 15px 0 5px; }
    dl dt:first-child {
      padding: 0; }
    dl dt > :first-child {
      margin-top: 0; }
    dl dt > :last-child {
      margin-bottom: 0; }
  dl dd {
    margin: 0 0 15px;
    padding: 0 15px; }
    dl dd > :first-child {
      margin-top: 0; }
    dl dd > :last-child {
      margin-bottom: 0; }

blockquote {
  border-left: 4px solid #dddddd;
  padding: 0 15px;
  color: #777777; }
  blockquote > :first-child {
    margin-top: 0; }
  blockquote > :last-child {
    margin-bottom: 0; }

table {
  padding: 0;border-collapse: collapse; }
  table tr {
    border-top: 1px solid #cccccc;
    background-color: white;
    margin: 0;
    padding: 0; }
    table tr:nth-child(2n) {
      background-color: #f8f8f8; }
    table tr th {
      font-weight: bold;
      border: 1px solid #cccccc;
      margin: 0;
      padding: 6px 13px; }
    table tr td {
      border: 1px solid #cccccc;
      margin: 0;
      padding: 6px 13px; }
    table tr th :first-child, table tr td :first-child {
      margin-top: 0; }
    table tr th :last-child, table tr td :last-child {
      margin-bottom: 0; }

img {
  max-width: 100%; }

span.frame {
  display: block;
  overflow: hidden; }
  span.frame > span {
    border: 1px solid #dddddd;
    display: block;
    float: left;
    overflow: hidden;
    margin: 13px 0 0;
    padding: 7px;
    width: auto; }
  span.frame span img {
    display: block;
    float: left; }
  span.frame span span {
    clear: both;
    color: #333333;
    display: block;
    padding: 5px 0 0; }
span.align-center {
  display: block;
  overflow: hidden;
  clear: both; }
  span.align-center > span {
    display: block;
    overflow: hidden;
    margin: 13px auto 0;
    text-align: center; }
  span.align-center span img {
    margin: 0 auto;
    text-align: center; }
span.align-right {
  display: block;
  overflow: hidden;
  clear: both; }
  span.align-right > span {
    display: block;
    overflow: hidden;
    margin: 13px 0 0;
    text-align: right; }
  span.align-right span img {
    margin: 0;
    text-align: right; }
span.float-left {
  display: block;
  margin-right: 13px;
  overflow: hidden;
  float: left; }
  span.float-left span {
    margin: 13px 0 0; }
span.float-right {
  display: block;
  margin-left: 13px;
  overflow: hidden;
  float: right; }
  span.float-right > span {
    display: block;
    overflow: hidden;
    margin: 13px auto 0;
    text-align: right; }

code, tt {
  margin: 0 2px;
  padding: 0 5px;
  white-space: nowrap;
  border: 1px solid #eaeaea;
  background-color: #f8f8f8;
  border-radius: 3px; }

pre code {
  margin: 0;
  padding: 0;
  white-space: pre;
  border: none;
  background: transparent; }

.highlight pre {
  background-color: #f8f8f8;
  border: 1px solid #cccccc;
  font-size: 13px;
  line-height: 19px;
  overflow: auto;
  padding: 6px 10px;
  border-radius: 3px; }

pre {
  background-color: #f8f8f8;
  border: 1px solid #cccccc;
  font-size: 13px;
  line-height: 19px;
  overflow: auto;
  padding: 6px 10px;
  border-radius: 3px; }
  pre code, pre tt {
    background-color: transparent;
    border: none; }

sup {
    font-size: 0.83em;
    vertical-align: super;
    line-height: 0;
}

kbd {
  display: inline-block;
  padding: 3px 5px;
  font-size: 11px;
  line-height: 10px;
  color: #555;
  vertical-align: middle;
  background-color: #fcfcfc;
  border: solid 1px #ccc;
  border-bottom-color: #bbb;
  border-radius: 3px;
  box-shadow: inset 0 -1px 0 #bbb
}

* {
	-webkit-print-color-adjust: exact;
}
@media screen and (min-width: 914px) {
    body {
        width: 854px;
        margin:0 auto;
    }
}
@media print {
	table, pre {
		page-break-inside: avoid;
	}
	pre {
		word-wrap: break-word;
	}
}
</style>


</head>

<body>

<h1 id="toc_0">Redis Cluster 集群环境搭建、测试</h1>

<h3 id="toc_1">目录</h3>

<ul>
<li><a href="#jump1">Redis Cluster 架构、特性</a></li>
<li><a href="#jump2">环境搭建</a>

<ul>
<li><a href="#jump3">Redis 安装包下载、解压、安装</a></li>
<li><a href="#jump4">redis.conf 配置文件</a></li>
<li><a href="#jump5">集群节点开启</a></li>
<li><a href="#jump6">配置Redis Cluster ，建立集群</a></li>
</ul></li>
<li><p><a href="#jump7">测试</a></p>

<ul>
<li><a href="#jump8">手动编码</a></li>
<li><a href="#jump9">redis-benchmark</a></li>
</ul>

<hr></li>
</ul>

<blockquote>
<p>先看 Redis Cluster 架构、特性，做简单了解，</p>

<p>接下来的通过集群搭建实例进行深入</p>
</blockquote>

<p><strong><em><spen id="jump1">先上架构图：</spend></em></strong><img src="http://i2.bvimg.com/616265/7a9ccfb7e70a9e73s.png" alt="Markdown"></p>

<ul>
<li>架构细节:</li>
<li>(1)所有的redis节点彼此互联(PING-PONG机制),内部使用二进制协议优化传输速度和带宽.</li>
<li>(2)节点的fail是通过集群中超过半数的master节点检测失效时才生效.</li>
<li>(3)客户端与redis节点直连,不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可</li>
<li>(4)redis-cluster把所有的物理节点映射到[0-16383]slot上,cluster 负责维护node&lt;-&gt;slot&lt;-&gt;key</li>
</ul>

<p><strong><em>可用性部分</em></strong>：</p>

<p>Redis Cluster 提供了选举机制，在主节点不可用的情况下，<strong>在从节点中选举出新的主节点</strong>，并移动哈希桶，使其替代主节点提供服务，提高集群的可用性。
<img src="http://i4.bvimg.com/616265/4e638f2493a4969ds.png" alt="Markdown"></p>

<ul>
<li>选举过程是集群中所有master参与,如果半数以上master节点与故障节点通信超过(cluster-node-timeout),认为该节点故障，自动触发故障转移操作.</li>
<li>当发生以下情况，则集群不可用：</li>
</ul>

<blockquote>
<ol>
<li>如果集群任意master挂掉,且当前master没有slave.集群进入fail状态,也可以理解成集群的slot映射[0-16383]不完成时进入fail状态. ps : redis-3.0.0.rc1加入cluster-require-full-coverage参数,默认关闭,打开集群兼容部分失败.</li>
<li>如果集群超过半数以上master挂掉，无论是否有slave集群进入fail状态.</li>
</ol>

<p>**当集群不可用时,所有对集群的操作做都不可用，收到(error) CLUSTERDOWN The 
  cluster is down </p>
</blockquote>

<hr>

<h1 id="toc_2"><spend id="jump2">搭建流程</spend>：</h1>

<blockquote>
<ol>
<li>Redis 安装包下载、解压、安装</li>
<li>redis.conf 配置文件</li>
<li>集群节点开启</li>
<li>配置Redis Cluster ，建立集群</li>
<li>集群测试</li>
</ol>
</blockquote>

<h2 id="toc_3"><spend id="jump3">1.Redis 安装包下载、解压、安装</spend></h2>

<ul>
<li>Linux环境下：</li>
</ul>

<div><pre><code class="language-none">wget http://download.redis.io/releases/redis-3.2.8.tar.gz</code></pre></div>

<ul>
<li><p>手动下载
<a href="http://download.redis.io/releases/redis-3.2.8.tar.gz">点此下载:redis_3.2.8</a></p></li>
<li><p>各个版本的redis下载地址：
<a href="http://download.redis.io/releases/">http://download.redis.io/releases</a></p></li>
</ul>

<hr>

<p>下载完成后进行解压并放置合适位置，因为集群环境，文件需要注意区分，加上端口号以标识，默认情况下，redis服务的端口号为6379</p>

<div><pre><code class="language-none">tar -zxvf redis-3.2.8.tar.gz
mv redis-3.2.8 ~/redis-3.2.8-6379</code></pre></div>

<p>避免权限不足导致的问题（此处放置在个人文件夹下，如果有多个用户操作需要注意），可以设置解压文件的读取权限为</p>

<div><pre><code class="language-none">sudo chmod -R 777 redis-3.2.8-6379 </code></pre></div>

<hr>

<p>运行安装</p>

<div><pre><code class="language-none">make &amp;&amp; make install</code></pre></div>

<p>至此redis安装包解压、安装完毕，准备配置文件并启动redis服务器
<strong>注：</strong>运行安装后可直接复制redis文件夹内容，不需要重复安装、构建</p>

<h2 id="toc_4"><spend id="jump4">2.redis.conf 配置文件</spend></h2>

<p>config文件在之前解压的redis-3.2.8-6379 文件夹内，可以对默认的配置文件进行备份</p>

<div><pre><code class="language-none">cd redis-3.2.8-6379 
cp redis.conf redis.conf.bac</code></pre></div>

<p>如果没有可以自行创建并贴入配置内容，配置内容在后面</p>

<div><pre><code class="language-none">cd redis-3.2.8-6379 
vim redis.conf</code></pre></div>

<p>配置完成后 ESC键 + :wq 保存</p>

<hr>

<p><a href="http://blog.csdn.net/xiaolong2w/article/details/8224839">===&gt;Linux下vim相关操作请移步&lt;===</a></p>

<hr>

<p>配置文件详细：Redis Cluster 会在主节点宕机后从主节点的从节点中选举出节点替代主节点，需要Redis Cluster 主节点宕机后快速恢复继续提供服务，<strong>请注意配置中的</strong></p>

<div><pre><code class="language-none">cluster-require-full-coverage no</code></pre></div>

<p>设置为关闭，否则有哈希桶覆盖的节点宕机时集群会<strong>拒绝服务</strong></p>

<div><pre><code class="language-none">#redis.conf默认配置
daemonize yes


##多实例情况下需修改，例如redis_6380.pid
pidfile /Users/mac/var/run/redis/redis_6379.pid  


#多实例情况下需要修改,例如6380
port 6379　　　　　　　　
tcp-backlog 511
bind 127.0.0.1
timeout 0
tcp-keepalive 0
loglevel notice

#多实例情况下需要修改，例如6380.log
logfile /Users/mac/var/log/redis/redis_6379.log
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes

#多实例情况下需要修改，例如dump.6380.rdb
dbfilename dump_6379.rdb

slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly yes
#多实例情况下需要修改,例如 appendonly_6380.aof
appendfilename &quot;appendonly_6379.aof&quot;
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events &quot;&quot;
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10

#################自定义配置
#系统配置
#vim /etc/sysctl.conf
#vm.overcommit_memory = 1

aof-rewrite-incremental-fsync yes
maxmemory 4096mb
maxmemory-policy allkeys-lru
#多实例情况下需要修改，例如/data/6380
dir /Users/mac/opt/redis/data/6379/

#集群配置
cluster-enabled yes
cluster-require-full-coverage no

#多实例情况下需要修改，例如/6380/
cluster-config-file /Users/mac/opt/redis/data/6379/nodes.conf
cluster-node-timeout 5000


#从ping主间隔默认10秒
#复制超时时间
#repl-timeout 60

#远距离主从
#config set client-output-buffer-limit &quot;slave 536870912 536870912 0&quot;
#config set repl-backlog-size 209715200
</code></pre></div>

<p><strong>注：</strong></p>

<ol>
<li><p>配置文件中参与的各个文件，除文件本身不需要手动创建外，文件路径需要<strong>手动创建</strong>，否则会在控制台或log文件中报错，服务不能正确开启，创建例如下，可更改至个人目录下</p>

<p>mkdir /Users/mac/var/run/redis/
mkdir /Users/mac/var/log/redis/
mkdir /Users/mac/opt/redis/data/6379/</p></li>
<li><p>配置文件文件中描述的和单节点有关的配置信息，请注意做好区分</p></li>
</ol>

<hr>

<p>需要复制 redis-3.2.8-6379 多份至同目录下，并做好区分、修改配置文件内容（端口号、日志文件、本地存储文件、Redis Cluster 节点配置文件等）</p>

<div><pre><code class="language-none">cp redis-3.2.8-6379 redis-3.2.8-6380
cp redis-3.2.8-6379 redis-3.2.8-6382
...</code></pre></div>

<p>推荐至少 6 节点 ，采用 3主+3从 方案</p>

<hr>

<p>至此，单个节点的 redis.conf 已经配置为工作在 Redis Cluster下的模式</p>

<h2 id="toc_5"><spend id="jump5">3.集群节点开启</spend></h2>

<p>开启各个节点，并检查开启情况</p>

<div><pre><code class="language-none">cd redis-3.2.8-6379/src
##以之前各个节点的redis.conf文件为配置运行启动
./redis-server ../redis.conf</code></pre></div>

<p>各个节点执行成功后，查看节点运行情况</p>

<div><pre><code class="language-none">ps -ef | grep redis

----------回车后查看，检查自己开启的各个节点---------
---因为测试运行后的宕机情况，此处只有四个节点在运行----

501 55589     1   0  7:10下午 ??         1:12.23 ./redis-server 127.0.0.1:6381 [cluster]
501 55599     1   0  7:11下午 ??         1:08.34 ./redis-server 127.0.0.1:6382 [cluster]
501 55608     1   0  7:11下午 ??         0:58.76 ./redis-server 127.0.0.1:6383 [cluster]
501 55616     1   0  7:11下午 ??         0:58.31 ./redis-server 127.0.0.1:6384 [cluster]</code></pre></div>

<p>贴上6379端口 redis 的日志文件</p>

<div><pre><code class="language-none">66322:M 01 Nov 00:04:16.091 * Increased maximum number of open files to 10032 (it was originally set to 256).
66322:M 01 Nov 00:04:16.093 * No cluster configuration found, I&#39;m 6ee9326104583183cde39aef3590977a6c027d42
                _._                                                  
           _.-``__ &#39;&#39;-._                                             
      _.-``    `.  `_.  &#39;&#39;-._           Redis 3.2.8 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ &#39;&#39;-._                                   
 (    &#39;      ,       .-`  | `,    )     Running in cluster mode
 |`-._`-...-` __...-.``-._|&#39;` _.-&#39;|     Port: 6385
 |    `-._   `._    /     _.-&#39;    |     PID: 66322
  `-._    `-._  `-./  _.-&#39;    _.-&#39;                                   
 |`-._`-._    `-.__.-&#39;    _.-&#39;_.-&#39;|                                  
 |    `-._`-._        _.-&#39;_.-&#39;    |           http://redis.io        
  `-._    `-._`-.__.-&#39;_.-&#39;    _.-&#39;                                   
 |`-._`-._    `-.__.-&#39;    _.-&#39;_.-&#39;|                                  
 |    `-._`-._        _.-&#39;_.-&#39;    |                                  
  `-._    `-._`-.__.-&#39;_.-&#39;    _.-&#39;                                   
      `-._    `-.__.-&#39;    _.-&#39;                                       
          `-._        _.-&#39;                                           
              `-.__.-&#39;                                               

66322:M 01 Nov 00:04:16.096 # Server started, Redis version 3.2.8
66322:M 01 Nov 00:04:16.096 * The server is now ready to accept connections on port 6385</code></pre></div>

<hr>

<p>至此，可见6379上的 redis 服务已经正常以 Cluster 模式运行，需要重复执行上述操作，直至开启所有的节点</p>

<p><strong>注：</strong></p>

<ol>
<li>节点开启不成功可能是文件路径不存在或错误造成的，请仔细检查log文件或命令行中的提示</li>
<li>节点的主从配置在 redis Cluster 初始化的时候由 Cluster 自动分配，在此只需要成功开启节点即可</li>
<li>可以写个脚本自动开启各个节点,vim 编辑后存为start-all.sh</li>
</ol>

<div><pre><code class="language-none">cd /Users/mac/redis-3.2.8/src
./redis-server ../redis.conf
...</code></pre></div>

<p>修改脚本权限</p>

<div><pre><code class="language-none">sudo chmod 777 start-all.sh   </code></pre></div>

<h2 id="toc_6"><spend id="jump6">4.配置Redis Cluster ，建立集群</spend></h2>

<p>所有服务正常开启并确认后，开始建立集群</p>

<p>命令： redis-trib.rb 此为官方提供的 ruby 脚本，在 redis-server 同目录下，用来管理集群，需要 ruby 环境依赖</p>

<p><a href="http://www.runoob.com/ruby/ruby-installation-unix.html">===&gt;Linux下安装 Ruby环境 &lt;===</a></p>

<p><a href="http://www.heatware.net/ruby-rails/how-to-install-rubygems-linux-ubuntu-10/">===&gt;Linux下安装 RubyGems &lt;===</a></p>

<p>下载安装后，需要在 gem 中安装 redis 组件
<code>
gem install redis --version 3.0.0  
</code></p>

<p>安装完毕后进行cluster集群搭建</p>

<div><pre><code class="language-none">##在任意节点内执行皆可，不需要在各节点内重复执行
cd redis-3.2.8-6379/src
###后接host:port 需要列出之前开启的各个节点，中间空格分隔
./redis-trib.rb create --replicas 1 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:6382 127.0.0.1:6383 127.0.0.1:6384</code></pre></div>

<ul>
<li>给定 redis-trib.rb 程序的命令是 create ， 这表示创建一个新的集群。</li>
<li>选项 --replicas 1 表示为集群中的每个主节点创建一个从节点。</li>
<li>之后跟着的其他参数则是实例的地址列表，程序使用这些地址所指示的实例来创建新集群。</li>
</ul>

<h2 id="toc_7"></h2>

<p>以上命令的意思就是让 redis-trib 程序创建一个包含三个主节点和三个从节点的集群。</p>

<p>接着， redis-trib 会打印出一份预想中的配置</p>

<div><pre><code class="language-none">
&gt;&gt;&gt; Creating cluster
Connecting to node 127.0.0.1:6379: OK
Connecting to node 127.0.0.1:6380: OK
Connecting to node 127.0.0.1:6381: OK
Connecting to node 127.0.0.1:6382: OK
Connecting to node 127.0.0.1:6383: OK
Connecting to node 127.0.0.1:6384: OK
&gt;&gt;&gt; Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:6379
127.0.0.1:6380
127.0.0.1:6381
Adding replica 127.0.0.1:6382 to 127.0.0.1:6379
Adding replica 127.0.0.1:6383 to 127.0.0.1:6380
Adding replica 127.0.0.1:6384 to 127.0.0.1:6381
M: 2774f156af482b4f76a5c0bda8ec561a8a1719c2 127.0.0.1:6380
   slots:0-5460 (5461 slots) master
M: 2d03b862083ee1b1785dba5db2987739cf3a80eb 127.0.0.1:6380
   slots:5461-10922 (5462 slots) master
M: 0456869a2c2359c3e06e065a09de86df2e3135ac 127.0.0.1:6381
   slots:10923-16383 (5461 slots) master
S: 37b251500385929d5c54a005809377681b95ca90 127.0.0.1:6382
   replicates 2774f156af482b4f76a5c0bda8ec561a8a1719c2
S: e2e2e692c40fc34f700762d1fe3a8df94816a062 127.0.0.1:6383
   replicates 2d03b862083ee1b1785dba5db2987739cf3a80eb
S: 9923235f8f2b2587407350b1d8b887a7a59de8db 127.0.0.1:6384
   replicates 0456869a2c2359c3e06e065a09de86df2e3135ac
Can I set the above configuration? (type &#39;yes&#39; to accept): 
</code></pre></div>

<p>输入 yes 并按下回车确认之后，集群就会将配置应用到各个节点，并连接起（join）各个节点，让各个节点开始互相通讯：</p>

<div><pre><code class="language-none">Can I set the above configuration? (type &#39;yes&#39; to accept): yes
&gt;&gt;&gt; Nodes configuration updated
&gt;&gt;&gt; Assign a different config epoch to each node
&gt;&gt;&gt; Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join......
&gt;&gt;&gt; Performing Cluster Check (using node 127.0.0.1:6379)
S: 8c4adebd6f91e6c1ea4a0994969eba712835df84 127.0.0.1:6379
   slots: (0 slots) slave
   replicates efa18d25418e75fd11ec95d30ffb7a8403d2acf1
M: 2fc4f6a4d1f515bdf549409c85d502a247eebe52 127.0.0.1:6381
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 144d631c4fe8386e55e2b3be0bdb72cfbbe1469b 127.0.0.1:6383
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: f773c40f5f61e289a9a2f641afe2aa7dc6d8099e 127.0.0.1:6380
   slots: (0 slots) slave
   replicates 144d631c4fe8386e55e2b3be0bdb72cfbbe1469b
M: efa18d25418e75fd11ec95d30ffb7a8403d2acf1 127.0.0.1:6382
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 2ca2e4f8553b24c814f7f4120fe7256620e9cf92 127.0.0.1:6384
   slots: (0 slots) slave
   replicates 2fc4f6a4d1f515bdf549409c85d502a247eebe52
[OK] All nodes agree about slots configuration.
&gt;&gt;&gt; Check for open slots...
&gt;&gt;&gt; Check slots coverage...
[OK] All 16384 slots covered.</code></pre></div>

<h2 id="toc_8"></h2>

<p>至此集群搭建已经完成。</p>

<blockquote>
<p>集群状态监测可以通过 
<code>./redis-trib.rb check ip:port</code>
连接任一集群内节点，查看集群各个节点状态</p>

<div><pre><code class="language-none">&gt;&gt;&gt; Performing Cluster Check (using node 127.0.0.1:6379)
S: 8c4adebd6f91e6c1ea4a0994969eba712835df84 127.0.0.1:6379
   slots: (0 slots) slave
   replicates efa18d25418e75fd11ec95d30ffb7a8403d2acf1
M: 6ee9326104583183cde39aef3590977a6c027d42 127.0.0.1:6385
   slots:0-1364,5461-6826,10923-12287 (4096 slots) master
   0 additional replica(s)
M: 2fc4f6a4d1f515bdf549409c85d502a247eebe52 127.0.0.1:6381
   slots:12288-16383 (4096 slots) master
   1 additional replica(s)
M: 144d631c4fe8386e55e2b3be0bdb72cfbbe1469b 127.0.0.1:6383
   slots:6827-10922 (4096 slots) master
   1 additional replica(s)
S: f773c40f5f61e289a9a2f641afe2aa7dc6d8099e 127.0.0.1:6380
   slots: (0 slots) slave
   replicates 144d631c4fe8386e55e2b3be0bdb72cfbbe1469b
M: efa18d25418e75fd11ec95d30ffb7a8403d2acf1 127.0.0.1:6382
   slots:1365-5460 (4096 slots) master
   1 additional replica(s)
S: 2ca2e4f8553b24c814f7f4120fe7256620e9cf92 127.0.0.1:6384
   slots: (0 slots) slave
   replicates 2fc4f6a4d1f515bdf549409c85d502a247eebe52
[OK] All nodes agree about slots configuration.
&gt;&gt; Check for open slots...
&gt;&gt; Check slots coverage...
[OK] All 16384 slots covered.</code></pre></div>

<p>单个节点的状态可以通过（连接任一集群内的节点皆可）</p>

<div><pre><code class="language-none">./redis-cli ip:port
127.0.0.1:6379&gt;info
# Server
redis_version:3.2.8
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:2391aacb4e8248f9
redis_mode:cluster
os:Darwin 17.0.0 x86_64
arch_bits:64
multiplexing_api:kqueue
gcc_version:4.2.1
process_id:67261
run_id:8e28c098ef11ade7e18c9862e5051ace2bdcb073
tcp_port:6379
uptime_in_seconds:34482
uptime_in_days:0
hz:10
lru_clock:16330206
executable:/Users/mac/redis-3.2.8-6379/src/./redis-server
config_file:/Users/mac/redis-3.2.8-6379/redis.conf
# Clients
connected_clients:2
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
# Memory
used_memory:1504496
used_memory_human:1.43M
used_memory_rss:1777664
used_memory_rss_human:1.70M
used_memory_peak:1504592
used_memory_peak_human:1.43M
total_system_memory:8589934592
total_system_memory_human:8.00G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:4294967296
maxmemory_human:4.00G
maxmemory_policy:allkeys-lru
mem_fragmentation_ratio:1.18
mem_allocator:libc
# Persistence
loading:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1509501397
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:0
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_current_size:207
aof_base_size:128
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0
# Stats
total_connections_received:7
total_commands_processed:346
instantaneous_ops_per_sec:0
total_net_input_bytes:5282
total_net_output_bytes:12045240
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.05
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:296
migrate_cached_sockets:0
# Replication
role:slave
master_host:127.0.0.1
master_port:6382
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_repl_offset:6007901
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:2.55
used_cpu_user:1.63
used_cpu_sys_children:0.00
used_cpu_user_children:0.00
# Cluster
cluster_enabled:1</code></pre></div>
</blockquote>

<h2 id="toc_9"><spend id="jump7">集群测试</spend></h2>

<blockquote>
<p>集群测试采用两种方式</p>

<ol>
<li>手动编码</li>
<li>redis-benchmark</li>
</ol>
</blockquote>

<h3 id="toc_10"><spend id="jump8">5.1手动编码测试</spend></h3>

<div><pre><code class="language-none">import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
import redis.clients.jedis.JedisPoolConfig;

import java.util.HashSet;
import java.util.Set;
import java.util.UUID;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

/**
 *
 * @author daniel
 * @email 576699909@qq.com
 * @time 2016-3-29 下午3:41:31
 */
public class RedisTest {
    private static final Log log = LogFactory.getLog(RedisTest.class);

    public static void main(String[] args) {
        RedisTest test1=new RedisTest();
        test1.doTest();
    }

    public void doTest(){
        // 数据库链接池配置
        JedisPoolConfig config = new JedisPoolConfig();
        config.setMaxTotal(100);
        config.setMaxIdle(50);
        config.setMinIdle(20);
        config.setMaxWaitMillis(6 * 1000);
        config.setTestOnBorrow(true);

        // Redis集群的节点集合
        Set&lt;HostAndPort&gt; jedisClusterNodes = new HashSet&lt;HostAndPort&gt;();
        jedisClusterNodes.add(new HostAndPort(&quot;127.0.0.1&quot;, 6379));
        jedisClusterNodes.add(new HostAndPort(&quot;127.0.0.1&quot;, 6380));
        jedisClusterNodes.add(new HostAndPort(&quot;127.0.0.1&quot;, 6381));
        jedisClusterNodes.add(new HostAndPort(&quot;127.0.0.1&quot;, 6382));
        jedisClusterNodes.add(new HostAndPort(&quot;127.0.0.1&quot;, 6383));
        jedisClusterNodes.add(new HostAndPort(&quot;127.0.0.1&quot;, 6384));

        ScheduledExecutorService scheduledThreadPool = Executors.newScheduledThreadPool(20);


        for(int i=0;i&lt;10;i++){
//            new Thread(new RedisRunable(jedisClusterNodes,config,i)).start();
            scheduledThreadPool.execute(new RedisRunable(jedisClusterNodes,config,i));
        }
        scheduledThreadPool.shutdown();
        double start=System.currentTimeMillis();
        try
        {
            // awaitTermination返回false即超时会继续循环，返回true即线程池中的线程执行完成主线程跳出循环往下执行，每隔10秒循环一次
            while (!scheduledThreadPool.awaitTermination(5, TimeUnit.SECONDS)){}
        }
        catch (InterruptedException e)
        {
            e.printStackTrace();
        }

        double end =System.currentTimeMillis();
        System.out.println((end-start)/1000+&quot;s&quot;);
    }

    class RedisRunable implements Runnable{

        private JedisPoolConfig config;

        private Set&lt;HostAndPort&gt; jedisClusterNodes;

        private int flag;

        RedisRunable(){}

        RedisRunable(Set&lt;HostAndPort&gt; jedisClusterNodes,JedisPoolConfig config,int i){
            this.config=config;
            this.jedisClusterNodes = jedisClusterNodes;
            this.flag=i;
        }

        @Override
        public void run() {
            while(true){
                try {
                    // 根据节点集创集群链接对象
                    //JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes);
                    // 集群各节点集合，超时时间，最多重定向次数，链接池
                    JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes, 5000, 100, config);
                    int num = 2000;
                    String key = &quot;name&quot;;
                    String value=&quot;&quot;;
                    int count = 1;

                    for (int i=1; i &lt;= num; i++) {
                        String uuid= UUID.randomUUID()+&quot;&quot;;
                        try {
                            // 存数据
//                        jedisCluster.set(key + i + &quot;&quot; + flag, &quot;yhj&quot; + i);


                            //存储随机id
                            jedisCluster.set(uuid, uuid+&quot;yhj&quot;+i);

                            // 取数据
//                        value = jedisCluster.get(key + i + &quot;&quot; + flag);
//                        log.info(Thread.currentThread()+&quot;&quot;+key + i + &quot;&quot; + flag + &quot;=&quot; + value);
//                        if (value == null || &quot;&quot;.equals(value)) {
//                            log.error(&quot;===&gt;break&quot; + key + i + &quot; value is null&quot;);
//                            break;
//                        }

                            //随机读写
                            value = jedisCluster.get(uuid);
                            log.info(Thread.currentThread()+&quot;&quot;+uuid + &quot;=&quot; + value);
                            if (value == null || &quot;&quot;.equals(value)) {
                                log.error(&quot;===&gt;break&quot; + key + i + &quot; value is null&quot;);
                                break;
                            }
                        } catch (Exception e) {
                            log.error(&quot;====&gt;&quot;, e);
                            Thread.sleep(3000);
                            continue;
                        }
                        // 删除数据
                        jedisCluster.del(uuid);
                        Thread.sleep(5);
                        //value = jedisCluster.get(uuid);
                        //log.info(key+i + &quot;=&quot; + value);
                    }
                } catch (Exception e) {
                    log.error(&quot;====&gt;&quot;, e);
                }
            }

        }
    }
}</code></pre></div>

<p>执行结果如下：</p>

<div><pre><code class="language-none">信息: Thread[pool-1-thread-6,5,main]fc95c5b7-0516-4121-b8b5-8d9e43e52988=fc95c5b7-0516-4121-b8b5-8d9e43e52988yhj1918
Oct 31, 2017 7:22:26 PM com.yhj.domain.RedisTest$RedisRunable run
信息: Thread[pool-1-thread-10,5,main]cc75cbd5-0d6f-46df-a6ac-6dc3b0e7e3fd=cc75cbd5-0d6f-46df-a6ac-6dc3b0e7e3fdyhj117
Oct 31, 2017 7:22:26 PM com.yhj.domain.RedisTest$RedisRunable run
信息: Thread[pool-1-thread-5,5,main]c853f3f1-a5ff-4f66-9b12-e82f9c56fd9f=c853f3f1-a5ff-4f66-9b12-e82f9c56fd9fyhj1884
Oct 31, 2017 7:22:26 PM com.yhj.domain.RedisTest$RedisRunable run
信息: Thread[pool-1-thread-6,5,main]81176e59-7d41-4d92-ba94-f876c88b34dd=81176e59-7d41-4d92-ba94-f876c88b34ddyhj1919
Oct 31, 2017 7:22:26 PM com.yhj.domain.RedisTest$RedisRunable run
信息: Thread[pool-1-thread-10,5,main]1c7ce0f0-0340-4907-9c4e-191e5d298307=1c7ce0f0-0340-4907-9c4e-191e5d298307yhj118
...</code></pre></div>

<p>考虑到实际生产环境和测试用平台差异，以下统计结果仅供参考</p>

<blockquote>
<p><strong>6*1 Master</strong></p>

<p>10线程，每个线程2000个key，一次插入一次读取，处理时间约7s，读写平均5600操作/s，</p>

<p>10线程，每个线程2000个key，单插入操作，处理时间约2.3s，大约写入速度 8700操作/s,</p>

<p>10线程，每个线程2000个key，单读取操作，处理时间约5.4s，大约读取速度 3700操作/s</p>

<p><strong>3 Master + 3 Slave</strong></p>

<p>10线程，每个线程2000个key，一次插入一次读取，处理时间约7.5s，读写平均5300操作/s，</p>

<p>10线程，每个线程2000个key，单插入操作，处理时间约2.3s，大约写入速度 7662操作/s,</p>

<p>10线程，每个线程2000个key，单读取操作，处理时间约6.2s，大约读取速度 3225操作/s</p>

<p>10线程，每个线程2000个key，一次插入一次读取一次删除，处理时间约10s，平均6000操作/s，</p>
</blockquote>

<hr>

<h3 id="toc_11"><spend id="jump9">5.2 redis-benchmark</spend></h3>

<blockquote>
<p>redis-benchmark 是官方提供的压力测试工具，可以模拟并发请求，测试比较理想化，实际生产中，结合网络、平台配置、编码等因素，结果有差异</p>
</blockquote>

<div><pre><code class="language-none">###100个并发连接，100000个请求，检测host为localhost 
###端口为6379的redis服务器性能
### -q 选项表示只显示 query/sec 的缩略信息

yanhuajun:src yanhuajun$ ./redis-benchmark -h 127.0.0.1 -p 6379 -c 100 -n 100000 -q
PING_INLINE: 100502.52 requests per second
PING_BULK: 99700.90 requests per second
SET: 94161.95 requests per second
GET: 97181.73 requests per second
INCR: 97943.19 requests per second
LPUSH: 81433.22 requests per second
RPUSH: 91157.70 requests per second
LPOP: 89686.10 requests per second
RPOP: 93109.87 requests per second
SADD: 91240.88 requests per second
SPOP: 82644.62 requests per second
LPUSH (needed to benchmark LRANGE): 80128.20 requests per second
LRANGE_100 (first 100 elements): 91575.09 requests per second
LRANGE_300 (first 300 elements): 87032.20 requests per second
LRANGE_500 (first 450 elements): 85178.88 requests per second
LRANGE_600 (first 600 elements): 73152.89 requests per second
MSET (10 keys): 65530.80 requests per second</code></pre></div>

<hr>

<hr>

<p>至此，应该已经通过 Redis Cluster 构建了一个 redis 集群环境，并进行了状态检查，和基础测试，下一篇将介绍，通过 redis-trib.rb 来管理节点、哈希桶。</p>




</body>

</html>
